Evaluating AI Usage for Evaluation Purpose

Improving Report Summarization

“We are drowning in information, while starving for wisdom. The world henceforth will be run by synthesizers, people able to put together the right information at the right time, think critically about it, and make important choices wisely.”

Edward Osborne Wilson

Current Challenge

Considering the numbers of published evaluation reports across the UN system, information retrieval and evidence generalization challenges have arisen:

How to extract the most relevant findings and recommendations from within a specific context and to reuse and re-inject them in a different but appropriate context?

The fifth wave of the evidence revolution is triggered by AI

The twenty-first century experimenting society: the four waves of the evidence revolution, published in Nature, Howard White, 2019

Results Cherry Picking: how to build effective “Evaluation Brief”?

Choosing what to include and what to exclude, especially in terms of highlighting critical aspects while deciding on what are the less relevant details to omit…

Relying on automated retrieval can help improving the objectivity and independence of the evaluation report summarization.

Cassandra, bearer of bad news

Retrieval-augmented generation (RAG) at rescue

Presentation title

Left column

Right column